The Expectatio Maximization Algorithm

common task in signal processing is the estimation of the parameters of a probability distribution function on. Perhaps the most frequently encountered estimation problem is the estimation of the mean of a signal in noise. In many parameter estimation problems the situation is more complicated because direct access to the data necessary to estimate the parameters is impossible, or some of the data are missing. Such difficulties arise when an outcome is a result of an accumulation of simpler outcomes, or when outcomes are clumped together, for example, in a binning or histogram operation. There may also be data dropouts or maximization) algorithm is ideally suited to problems of this sort, in that it produces maximum-likelihood (ML) estimates of parameters when there is a many-to-one mapping from an underlying distribution to the distribution goveming the observation. In this article, the EM algorithm is presented at a level suitable for signal processing practitioners who have had some exposure to estimation theory. (A brief summary of ML estimation is provided in Box 1 for review.) The EM algorithm consists of two major steps: an expectation step, followed by a maximization step. The expectation is with respect to the unknown underlying variables, using clustcrinr in such LI way that the niimhcr 01' undcrlying data points is unl\no\\ n (censor-ing ;tnd/or truncation). The FV (eq~ectation-the cui-wit c\tiniatc of the parametcrs ancl conilitioncil upon the observ;itions. The niaxinii/atiun \ ~ e p thcn provide\ a new e\ti